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1. Introduction 



H ■ 1.1. Symmetry Types. 



In [KS99], Katz and Sarnak introduced the notion of symmetry type for a family of L- 
functions. Since then there has been a slew of activity regarding the following problem: 
given a family of L-functions, determine its symmetry type. A common approach to this 
determination, as outlined in [Sar08], is to analyze the density of low- lying zeros in the 
specified family, for test functions whose Fourier transforms have restricted support. Such 
an analysis has been carried out in many places, including [ILSOO, RoyOl, Gul05, You06, 
HM07, AIL+11] for GL(2) and [DM06] for some GL(4) and GL(6) families; see also [DM09]. 
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In all cases in the literature (going beyond GL{1) or abelian methods), the analysis involves 
a version of the GL(2) Petersson/Kuznetsov formula. 

The purpose of this paper is to carry out a similar analysis, for the first time using the 
GL(3) Kuznetsov formula. Assuming the generalized Riemann hypothesis (to interpret the 
low-lying zero densities) and the generalized Ramanujan conjectures (for ease of exposition), 
we will determine symmetry types for the families of 

(1) cuspidal GL(3,Z) automorphic forms (f), 

(2) the GL(6) symmetric square family sym^ 0, and 

(3) the GL(8) adjoint family Ad(j). 

We will show that the symmetry types are: 

(1) unitary, 

(2) unitary, and 

(3) symplectic, 

respectively. 

The methods presented here are capable of wide generalization, in particular, it should 
be possible to determine the symmetry types of families associated to pairs of cuspidal 
automorphic representations on GL{n) for any n > 2. We hope to return to this topic in a 
future publication. 

To state our results more precisely, we need some background. 

1.2. Hecke-Maass forms. 

Let G = GL(3, M) with maximal compact K = 0(3) and center Z = M^, let l)^ = G/{K-Z) 
be the generalized upper half plane, and take the lattice F := GL(3, Z) in G. 

The algebra of G-invariant differential operators acts on H := L^(r\f)^). The Hecke-Maass 
forms (f)j constitute an orthogonal (Hecke normalized) basis for 

OD 

•Ho:=0C0,C7/, 

where "Hq is the cuspidal subspace in the Langlands spectral decomposition [G0IO6, Prop. 
10.13.1] 

/x ^ 'LI © Ho © iLmin © iLmax © rtres- 

Here T-Lmin, 'Umax, and Tires are, respectively, the spans of integrals of the minimal and 
maximal parabolic Eisenstein series, and the residual spectrum. 

Let the Hecke-Maass form (f)j have spectral parameters z/^-'^ := {i^i , 1^2 )■ When discussing 
a fixed form 0, we drop the superscripts (j). Our normalization^ is such that for a tempered 
form, Ui and z/2 are purely imaginary. 



""^Note that our normalization differs from tliat used in [G0IO6] by Vj i-> 1/3 
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It is convenient to also introduce the spectral parameters 

z/3 := z/i + 1^2, 

and 

«! := z/i + 1^3, ^2 := -1^1 + 1^2, "3 := -1^2 - i^3- 
Writing A,^ for the Laplace eigenvalue of 0, we have 

A^ = 1 - 3{i^^ + z/2 + I'D = l-{al + al + aj). 

Weyl's Law in this setting [MilOl] states that 

#{0 : A^ < T'} ~ cT', 

as T — )■ 00, for some constant c > 0. 

L3. The GL(3) Kuznetsov Formula. 

We will state and use the GL(3) Kuznetsov formula with some naturally occurring weights, 
defined as follows. For j = 1, 2, 3, . . . , let 

Cj := Res L(s, (pj x cpj^ 

be the residue at the edge of the critical strip of the L-function attached to (pj X(j)j; generically 
this is the value at s = 1 of L{s,Ad(j)j). 
We introduce an absolute constant 

R> 10, 
which is needed for certain technical reasons, see the estimates in 54.2. For T ::^ 1, we define 



^) 



■r-r -p /' 2+R+3iyj \ -p f 2+R-3uj 
l<j<3 ^ 



i<i<3 ^ ^ ^ ^ 

In fact, R is needed to enable us later to pull contours in certain integrals with respect to 
the z/j's without passing through poles of the numerator in (1.1). Note that hT,R{y) > 0, 
and is essentially supported on A < T^, or |z/i|, |j/2|, It's! ^ 7". In this range, one sees from 
Stirling's formula that if u is tempered, then 

hrA'^) ~ c^[(l + |z^i|)(l + W^Dil + l^^sl)]'', (1.2) 

for some cr > 0. The non-tempered forms constitute a zero density set [MilOl]. 

Let Aj{ni,n2) denote the coefficients of (j)j in the Fourier- Whittaker expansion, see §2.3. 
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Theorem 1.3. With the above notation and assuming the Ramanujan conjecture at the 
infinite place, we have the "Weyl Law", that for some a > 0, 

Moreover for fixed e > 0, R > 10, rii, 77,2, mi, ^2 ^ '^>i, o-nd T ^ 1, we have 



^ Aj{mi, m2)Aj{ni, Ui] 



hrA^ 



E ^^2v^ + Or^s r3+3«+- |mim2nin2p , tf '"^="^' 



■J 

OR^e\ T'^^^^^^ \mim2nin-2\^ j, otherwise. 

(1.5) 



Remark 1.6. In light of the asymptotic formula (1.2), the analytic weight hx.R can be removed 
with a modicum of effort; we have chosen to leave the weight for ease of exposition. The 
same is done for GL(2) in [AIL+11]. 

Remark 1.7. The weight Lj is more subtle; it is shown in [Bloll, (1.4)] that 

where 

C, = {l + \u^\){l + \u2\){l + \p^\). 

Moreover if one assumes the functorial transfer predicting x is automorphic on GL(9), 
then using the non-existence of Siegel zeros for the corresponding L-function [HR95], one can 
improve the lower bound above to C~^ . With this assumption, the weight can be removed 
completely, as in [LuoOl], giving rise to a clean cut-off. 

Remark 1.8. We have not made any attempt to obtain the best possible error terms in (1.5). 
In particular, we have made no use of stationary phase, nor have we even invoked Deligne's 
bounds for Kloosterman sums (see e.g. [BFG88, Larsen's appendix]). We tried to present 
as simple a method as we could, keeping in mind the eventual goal of generalizing these 
techniques to GL(n) with n>2. 

Remark 1.9. A similar result is obtained in [Bloll]. Blomer first chooses a test function 
on the geometric side, and then executes a delicate analysis to obtain implications on the 
spectral side. In our approach, we choose the test function on the spectral side first, making 
the asymptotic formula (1.2) immediately visible. In a private communication, Blomer has 
informed us that from the methods in [Bloll], he can also obtain (1.4) and (1.5) (with a 
better error term) for a range of test functions. 
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1.4. Low-Lying Zeros. 

For a Hecke-Maass form (f) on GL(3), let p^cf)) be one of 



P(0) 




and let L(s, p(0)) be the corresponding L-function. Let ai, 02, as be the spectral parameters 

associated to </>. If the Laplace eigenvalue A^ = 1 

we define the analytic conductor Cp(^ii,) of p(0) as follows. 



a\ + a\ + a1) is sufficiently large, then 



'tx-^ 



Cp(^) 



n ¥' 

1<A;<3 

l«fcl>5 



TT 



n 

i<i<fc<3 



l"j+"fe 



TT 



3 3 

nn 

i=i fc=i 



laj-Qfe 



if p(0) 



if p((;/)) = sym^ 



if p(0) = 0x0. 



;lio) 



Remark 1.11. Note that this is off by a constant from the more standard Iwaniec-Sarnak 
definition of "conductor," for which see e.g. [IK04, p. 95]. The constants are crucial in our 
applications, see specifically (7.7), so we make our definition as above. 

We are interested in the weighted average value, denoted Cp, of the conductor Cp(0) with 
respect to the weighting function /i^jj defined in (1.1). Then Cp is defined by 






hrA^ 



uh 



c, 



(T^oo) 



;i.i2) 



and satisfies 



C„ 



T3 ifp(0) = 0, 

T^ if p(</)) = sym^ 0, 

T6 ifp(</.) = Ad0. 

The weighted average value of the conductor in a family is introduced to normalize the low- 
lying zeros for comparison between the different families and the different matrix ensembles. 
Let ip be an even test function of Schwartz class on M and define the low-lying zeros sum 

log a 



D{p{<py,^):=Y,^ 



7 



2n 
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where 7 runs over the ordinates of nontrivial zeros of L{s, p{(f))), counted with multiphcity. To 
interpret this as capturing the low-lying zeros, we must assume GRH for the corresponding 
L-functions. As ip has rapid decay, this sum localizes to those 7 which are within 1/logCp 
of the origin (corresponding to the central point s = 1/2 of the L-function). 

Theorem 1.13. Assume the Fourier transform ip of ip has support in {—6,6), where 

[4/15, z/p(0)=0, 
6=l2/27, z/p(0)=sym2 0, 
[2/27, z/p(0)=Ad</.. 

Assume the Ramanujan conjectures, and GRH for the corresponding L-functions. Then we 
have the asymptotic formula 

■Y.D{p{cP,),^) ^^^^^^p =lj{x)W,^,){x)dx + o(^-^l^y (1.14) 



E. 



hT,R{^^^^) 



05 T —7- 00, with the limiting density function W above given by 

(1, ^/p((/') = 0, 

W^pw(^)=<1, ^/p((/>) = sym2 0, (1.15) 

That is, the family p{(f)) has symmetry type: unitary, unitary, and symplectic, respectively. 

Remark 1.16. The exterior square L-function on GL(3) is the same as the contragredient 
L-function (see [BF90, JS90, KonlO]). So the symmetry type for the exterior square family 
is unitary. 

Remark 1.17. Note that (1.15) is consistent with a recent conjecture by Shin and Templier 

[ST12]. 

Remark 1.18. As in Remark 1.8, the range of 6 above can also be improved, and is intimately 
tied to the error terms in (1.5). 

Remark 1.19. The Ramanujan conjectures are assumed to make the exposition of Theorem 
1.13 as simple as possible. They can easily be removed by decreasing the size of 6 in Theorem 
1.13. 

1.5. Outline. 

The rest of the paper is organized as follows. In §2, we collect various preliminaries 
on automorphic forms on GL3(Z) (their Fourier development and L-functions), and the 
Kontorovich-Lebedev-Whittaker transform, as explicated by the authors in [GKll]. In §3, 
we collect the GL(3) Kuznetsov formula, explicating all the terms which appear. 

The careful definition of the choice of test function is given in §4, where we also analyze 
its growth/decay properties; this is the most important and involved section. We note that, 
though the argument is a bit complicated (four- dimensional integrals of 12 Gamma factors 
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in the numerator and 7 Gamma factors in the denominator) , the analysis uses nothing more 
than Stirhng's asymptotics for the Gamma function. In §5, we input the estimates of §4 into 
the Kloosterman integrals appearing on the geometric side of the Kuznetsov formula, giving 
bounds for these, as well as estimating away the contribution from the Eisenstein spectrum. 
Combining all the above estimates, we prove Theorem 1.3 in §6. 

Next we turn our attention to the application to low-lying zeros. In §7, we develop the 
Explicit Formula for the various L- functions of interest, and analyze the local Langlands- 
Satake parameters in §8. Having done so, we apply Theorem 1.3 to the low- lying zeros sum 
in §9 to prove Theorem 1.13. 
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2. Preliminaries on Automorphic Forms on GLs{Z) 
2.1. Jacquet's Whittaker Function. 

Let 

()3:=GL3(M)/(03(M)xMX) 

denote the generalized upper lialf plane. For 2 G f)^ we use Iwasawa coordinates: 

/ 1 X2 2:3 \ / yiy2 

z = xy = \ 1 2:1 j yi 

\ 1 / \ 1 

where a;i,a;2,a;3 G M and yi,y2 > 0. We will frequently abuse notation, not distinguishing 
between y as above and y = (2/1,^2)- Equip f)^ with the Haar measure 

dxidx2dx'idyidy2 

{yiy2Y 

With this measure, the group F = Gh^ilj) is a lattice, that is, the quotient T\^^ has finite 
volume. In fact, the volume is 



^r. ^ ^. (2.1) 

r\f)3 
For the pair u = (z/i, V2) G C^, set 

1^3 := 1^1 + 1^2- (2.2) 

Then we have the /-function, defined by 

h{z) = iyw2Y-"''y?y2\ (z = xy e f)^). (2.3) 

We now define Jacquet's Whittaker function for GL^lM.).^ 

Definition 2.1 (Whittaker function). For z/ G C^ and z G f)'^, set 

3 



wHz) := --^'^^nr(^) 



l\ /l n2 ua^ 
x////,,|| 1 |( ImiI^I e{-ui t U2) duidu2du3. 

This function, originally defined for £Hez/i, 91ci^2 ^ 1, has analytic continuation to all 
1/ G C^. For z = y, the value of H^^(|/) is independent of the sign, so we drop the ±. 

It is convenient to define the parameters a, given in terms of z/, by the following linear 
relation: 

ai = 2z/i + 1/2, «2 = -1^1 + 1^2, "3 = -i^i - 2z/2- (2.4) 



Throughout we use the completed Whittaker function, in the terminology of [G0IO6] . 
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Then ai + 02 + 03 = and there is an action of the Weyl group which permutes the 
parameters ai, a2; c^s- We say that a function of ui, 1^2 is symmetric under the action of the 
Weyl group if it is invariant under all reorders of the triple (ai, 02, as). 

Consider the representation of the Whittaker function as a double inverse Mellin transform 
[StaOl] 

3/, , , n r (^) r (£Hzf^) 

^^(^) ^ w y y "a.^^^^^t{^) ^r-.rM..ci.2, (2.5) 

(Cl)(C2) 

for any Ci,C2 > 0. Here we use the standard convention that for C G M, the symbol iC) 
denotes the line C + iM. Note that Wy is symmetric under the action of the Weyl group. 
For s G C, Stade's formula [Sta02] gives 



<i,'c<3 



where /is = /xi + /i2 and Pi, 132, Pz are defined in terms of /ii,/i2 as in (2.4). The left side 
above is originally only defined for 9^e(s) sufficiently large; of course the right side gives its 
meromorphic continuation. 

2.2. Kontorovich-Lebedev transform. 

Next, we give the analogue of the Kontorovich-Lebedev transform for GL(3), often referred 
to as the Lebedev-Whittaker transform [GKll, Wal92]. Let / : M^ ^ C and define /" : 

C2 ^ C by 

/'M:=///(!/)H';fe)|^. (2.7) 

provided the integral converges absolutely. Then f^ is termed the Lebedev-Whittaker trans- 
form of /. Note that f^ inherits the property that it is symmetric under the action of the 
Weyl group. 

The inverse transform is given as follows. Assuming g is invariant under the action of the 
Weyl group and has sufficient decay, we define 

dh'idh'2 



A sufficient condition on the test functions above (see [GKll]) is that glu) have holomor- 
phic extension to a strip —77 < |9^e(i/i)|, |$Ke(z/2)| < V (for some r] > 0) and in this strip 
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satisfy 



\g{iy)\ <exp . 2^ ^fc 


)fl(l+-.lr°- 

' k- 1 


Then under these growth assumptions we have 




9 f ^^ 


/ = ^', 


and the Parseval-type relation: 




lOO JOO 

R2 — joo — joo 


, «. . C^i^lrf^/2 


j=i V / V / 



(2.9) 



(2.10) 



2.3. Cusp Forms. 

Take a Hecke-normahzed basis of Maass cusp forms {4>j}j=i,2,3... for 'Ho, the cuspidal 

(i) 1 /Q I ,.U)\ ..ru^^^ ,,(.7) _ ^,0) ,,0■)^ ^ 



,^^2 



subspace of L'^{T\i)^). The form 0^ is of type (l/3 + i^i , 1/3 + z/^^O where u^^'> = (i/[ 
C^ denote the spectral parameters. When speaking of a fixed Maass form (j), we drop the 
superscript (j). For tempered forms, the spectral parameters z/i and z/2 are purely imaginary. 
Then with 1^3 = lyi + 1^2, the Laplace eigenvalue A^ is related to u by 

X^ = 1 -6{u^ + u^ + ul). 

Each such has the Fourier- Whittaker development given by [Sha73, PS75, G0IO6]: 

7ec/2(z)\sL2(z)fci>ifc27^o ^1 21 yy ij ^ ^ J 

with the Hecke normalization A^(l, 1) = 1. 
The L-function attached to (j) is given by 

A{l,n) 



L{s,<P)--=J2 



n" 

n>l 



where A{l,n) = A^{l,n), i.e., we have dropped the from the notation. This constitutes a 
degree 3 L-function, which in completed form has Euler product 



A{s,(f)):=Y[Lp{s, 



with local factors for p < 00 of type 
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and for p = oo, 

The Rankin-Selberg L-function is 

\A{h,h)\^ 



L(s,0x0):=C(3s)5^ 



This L-function has a pole at s = 1. Standard Rankin-Selberg theory, together with Stade's 
formula (2.6) shows that the j-th Maass form (pj has L^ norm given by 

ll,,p.e.,.nr(i±|^)r(i^). ,2.12) 

where 

Cj := Ress=i L{s,(j)j x 0j). (2-13) 
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3. The GL(3) Kuznetsov Formula 

The following equation is the GL(3) Kuznetsov formula, as compiled from [BFG88] and 
[Bloll]: 

'C + £min + Sma.=M + ]C + JC + 1C\ I (3.1) 



where each component is explicated below. Let p : M^ — t- C be a test function with suitable 
decay properties; a sufficient condition is that 

|p(2/l,y2)|«(2/ly2)'+^ (3.2) 

^-s l/i)Z/2 -^ 0, and that p is otherwise bounded. Fix positive integers ni,n2,'mi,m2- 

The left hand side of (3.1), called the spectral side, consists of cuspidal and Eisenstein 
contributions. The cuspidal contribution is given by 

C = 2_^Aj{mi,m2)Aj{nun2) ^ y -^ y -y, (3-3) 



J 



QC, nr ^^^ r 



fc=i 



l+3i^"' \ in / l-3u 



^fc 



where the sum on j is over cuspidal Hecke-Maass forms (pj on GL(3,R). The minimal 
Eisenstein series contributes 

ioo joo 

^ ^'^ Joo-L n|c(i + 3^.)r(i±|^)f 

fc=l 
where the minimal Eisenstein coefficients satisfy 

\A^{ni,n2)\<^einin2y. (3.4) 

Lastly, the maximal Eisenstein contribution is 

B^.r^{mi, m2)By.r,{ni, n2) Pt,r y^ " "3"' ~^ J 



^'""^ Ott, Z^ 



27rz^ J L{l,Aduj)\L{l + 3iy,Uj 



-p I l+3i/— irj A p / l+2irj A p / l+Siy+ir.j 



du. 



where c is an absolute constant, and {uj} is a basis of Hecke-Maass forms for GL(2, Z), each 
of eigenvalue 1/4 + r|. The trivial bound for these Fourier coefficients is 

\B,,r,{nun2)\<^e{nin2Y/^^'. (3.5) 

Note that the residual spectrum does not contribute, having only degenerate terms in its 
Fourier expansion. 

For functions p,q : M^ — t- C let 

(p, q) = / / p{yu 2/2) q{yu 1/2) . ^ p - 
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Let 1(7 denote the indicator function, which is 1 if the condition C holds and otherwise. 
The right-hand side of (3.1), called the arithmetic side of the Kuznetsov formula, consists of 
a main term and Kloosterman contributions given by 

M = lr„^=„ji (p,p), (3.6) 

S{emi,ni,n2,Di,D2) ~ ( fnrn^mTX 



p. S:^ Sr^ oytini,ui,U2, 1^1,1^2) ^: 

^ = 2^ 2^ nTn: ^^ 



+ 1 n m ^1^2 VV D,D2 J 

e=±l -D1I-D2 

mo-Dr— ni Dn 



i2-^l="l-"2 



jrv ^ ST' \^ o{tiu2,n2,ni,j^2,^l) 7f 

h. h ^1^2 ^^vv D,D2 



S{em2,n2,ni,D2,Di) ~ ( \n\n2m2 



ii\-^ Z?;^— n2-Dx 



_ "sp V^ ^(eimi, e2m2, rii, r22, -Pi, A) ^ f y/m^n^D^ y/m2niD2 

^ ^ ^ /^l/^2 ^'''' V ^2 ' i^l 

ei,e2=±li5i,Z)2 ^ ^ \ ^ i 

Here S, S, J', J are certain GL(3) Kloosterman sums and integrals corresponding to various 
elements of the Weyl group. 

Let e[x) := e^'^*^. The Kloosterman sums are given explicitly by: 

Q( n n\ 1 \^\^ fmiCi + niCiC2\ ( n2C2 \ 
^(mi, ni, ns, I^i, D2) := 1di|D2 Z^ Z^ el 1 e I -jyjjy 1 , 



Cx (mod iJ]^ ) ,C2 (mod 132 ) 
(Ci,Di) = l = (C2,D2/I3l) 



and 



S{mi,m2,ni,n2,Di,D2) := Z^ Z^ Z^ Z^ ^1 ^ ) 



Bi,Ci(modDi) 
S2>C2(modD2) 

(i3l,Ci,Di) = l=(S2>C2,D2) 
BlS2+C'2-D2+C2Di=0(modOiD2) 

/m252 + ^2 (1^2/^1 -^2-61) 
X e ^ 

V D2 

where Yi, ^2, -Z^i, Z2 are determined by 

YiBi + ZiCi = l{modDi) and K^^s + ^2(^2 = 1 (mod ^2). 

The Kloosterman integrals are given by: 

V^l + X? + xl A y/l + xj 




J,{A) = A-' / / p{Ayi, 1/2) e{-eAxm) P 1/2 



1 + xf ' yiy2 1 + xj + xl 



X1X2 , A X2 \ , , C?Z/lC?Z/2 

^ e U/2 ■ r"^ — 2 "^ r"^ — —, — 2 "a:idx2 —, 

1 + xf i/i?/2 1 + xf + x^y vivi 
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and 

-2 




J,,^,^{Ai,A2) = {A1A2) ' p{Aiyi, A2y2)e{-eiAiXiyi - e2A2X2y2) 



A2 ^y {XiX2 - X-i)^ + XJ + 1 Ai y/xl + X^ + I 

X p I ■ 



X e - 



1/2 x| + x| + 1 ' yi (X1X2 - x^y + xi + i 

A2 0:1X3 + X2 Ai X2(xiX2 - xs) +a;i 



y2 x| + X2 + 1 2/1 (xiX2 - ^3)2 + x^ + 1^ 

^ , , , C?l/iC?2/2 

X dxidx2dx^ . 

2/12/2 
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4. Choice of Test Function and Bounds 

We now make a specific choice for tlie test function p{yi,y2)- By Lebedev-Wfiittaker 
inversion (2.8), we can just as well choose the transform p'^{i'i, ^2)- Let i? > 10 and T ^ 1. 
We define 

P^,«(^i, ^2) := v^ e^5^ n r 4 r ^ ^ . (4.1) 

This choice is motivated by the fact that we need p" = Ptr to be invariant under the action 
of the Weyl group, while also requiring cancellation of the exponential growth of the F- 
factors in the denominator on the right side of (3.3) (cuspidal contribution to the Kuznetsov 
formula). The variable i? > 10 is introduced to obtain absolute convergence of the sum (3.3), 
and to pull certain contours without passing through poles, see (4.16). Note first that pj^j^ 
easily satisfies the requisite bounds (2.9) for Lebedev-Whittaker inversion. It will be shown 
below that the inverse transform pt^r satisfies (3.2), see (4.19). 

Observe then that the cuspidal contribution (3.3) becomes 

C = 2_^Aj{mi,m2)Aj{ni,n2) -; , 

exactly as desired in (1.1). 

4.1. Some Auxiliary Bounds. 

We collect here some bounds coming from Stirling's asymptotic formula: 

\T{a + it)\^V2^\t\''-^e-^, (t ^ ±00), (4.2) 

for fixed o" G M. 

There are three types of integrals which we will need to estimate p^^. Throughout we 
have yi, ?/2 > 0, i? > 10, T > 1. 

The First Integral: 

For any Ci, C2 e M \ {-2, -4, -6, . . . }, let 

e^^'i^ n F(-i^)F(-^) 
l!ilUc„C2;y„y2) ■= 'T' , . : ^ (4-3) 



(0) (0) (C2) (Ci) n r (^^ j F y 



Sfi \ T^ I — Sp'j 



3 



n r (^) F (^) 
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The Second Integral: 

Similarly, for any hii,K2,Ci G M (so that the integrand below doesn't pass through poles 
of D, let 



X^^ij(Ki,K2,Ci;2/i,y2) •= / / / 



e" 


"l+"2+' 
2T2 


2 


i<i<3 


r| 


f 2+R+3uj ^ 


ri 


f2+R- 
K 4 


3uA 



(^.)(^o(co n r (^) r (:^) r (^) r (-1^) 

i=i 



The Third Integral: 



Lastly, for any Ki, /t:2 G M passing through no poles, let 



^rk^i'^^a; 2/1,^2) 



(K2) (ki) 



e' 


2 2 
2T2 


l<j<3 ^ 


^) 


p f2+R~3u^ 
\ 4 











r(¥) 


r(^)r 


(^ 


xyl" 


'"^ys"" 


"Ml^l 


dU2 



Define k[, Kj, fi:3 to be related to k's in the same way that a's are related to z/'s, that is, 
k[ = 2k,i + K2, ^2 = — fi^i + ^2, ^^3 = ~^i ~ 2/t:2- (4-4) 

Theorem 4.5. Fix R> 10 and e > 0. For any yi, y2 > and T ^ 1, we have the bound 

\I^fi^iC„C2;yuy2)\ «s,c.,c.,R {y^y2)T^^'^'^^' {y^y"' {^y"' r. (4.6) 

Moreover, 

\l!^^^{^u^2,Cr;yr,y2)\ «e,c„«„.2,K {yiy2)T'^'''^' (|)"^' (|)^''^'- ^^'^^ 

And finally, 

\T?U-u-2;yuy2)\ «s,..,..,r {ym)^''^'^" (|)"''^ (|)'''^T^ (4.8) 

We give separate treatments of each statement. 

Proof of (4.6). 

Write Vj = itj and Sj = Cj + iuj. The first exponential in the integrand gives arbitrary 
decay once \tj\ > T^"*"*^ for any e. Bringing the absolute values inside and applying Stirling's 
asymptotic formula gives 



l4!kCi.C'2; 2/1,^/2)1 <^CuC,,R,e yl'^'yl'"^' // V ■ exp (^^ ■ £yuidu2dhdt 



2, 
J J ^4 / 

ii|,|i2|<Ti+s 



J|(l + |afc + m2|) l + lm + nal 



') 
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where £ = £(ti,t2,Ui,U2) is the exponential factor: 

3 3 3 

£ = 3^\tk\-^\ak-iui\-^\ak + iu2\ + \ui + U2\, 
fc=i fc=i fc=i 

and V = Vc\,C2,R{'tiyt2, Ui, U2) is the polynomial factor: 

/ 3 \ («+2)/2 / 3 X (Ci-l)/2 

V = n(i+i4i) n(i+iafc-mii) 

\A:=1 / \A;=1 

/ 3 \ (C2-l)/2 / ^ (i_Ci-C2)/2 

\fc=l 

Note that we always have 

£ <0, 
with equality only when 

t2 — ti < ui < 2ti + ^2 and ti — t2 < U2 < ti + 2^2 

or 

— ti — 2^2 < Ml < ^2 — ^1 and — 2ti ~ t2 < U2 < ti — ^2- 

Hence, there is arbitrary decay outside of this range. Both inequalities have the same 
contribution, so we only deal with the second. 
Make a linear change variables 

ui h^ ui — ti — 2^2 and U2^^ U2 — 2ti — t2, 

so the new range is 

< ui < 3t2 and < ^2 < 3ti, (4.9) 

and the V factor becomes 

V, := ((l + |ti|)(l + |t2|)(l + |ti + t2|))^''+')/' 

X ((1 + Iml) (1 + |3ti + 3t2 - nil) (1 + |ni - 3t2|))^^'"'^^' 
X ((1 + \u2\) (1 + |3ti + 3t2 - M2I) (1 + \U2 - 3ti|))(^^-')/' 

X (1 + |-3ti - 3t2 +Ui+ U2\f~'''~^''^^^ . 

The integral of Vi over (4.9) in Ui,U2 is bounded up to constant by 

V2 := (l + |ti|)(^+^)/'+^Ml + |i2|)^''+'^/'+'^Ml + 1^1 + ^2!)^''+'^/'. 
Integrating V2 over the range \tj\ < T^"*""^ gives (4.6), as claimed. D 



Next we give a 
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Proof of (4.7). 

Again by Stirling's formula, we have 

l^Tjii'^i^ K2, Ci; yi, y2)\ <CiAi,«2Ae vl'^^^yl^"^' / / / '^ ' ^^P (4 ' ^) duidtidt2, 

|ti|,|i2|<ri+=IR 

where S = S{ti,t2,ui) is now the exponential factor: 

S = - |ti - t2 + ^ii| - 1^1 + 2^2 + '«i| + 3|t2|, 
and V = 'Pci,C2,Ki,K2,R(ti^t2,Ui) is now the polynomial factor: 

X (1 + |ti - t2 + nil) |(^i+-i-2-l) ^1 ^ 1^^ + 2t2 + nil) ^(Ci+-i+2«2-l)_ 

Note that we always have 

£ <0, 
with equality only when 

-ti - 2^2 < Ui < t2 - tl, 

SO we may restrict the Ui integral to this range. 
Make a linear change variables 

Ml I—)- Ml — ti — 2^2, 

so the new range is 

< ?ii < 3t2, 
and the V factor becomes 

V, := (1 + ItsI) ^''•+')/' (1 + |tl|) ("3«i+i?+l)/2 (1 + 1^^ + ^^1) (-3M-3K2+fl+l)/2 
X (|mi| + 1) (Cl+Ki+2«2-l)/2 (1^^ _ 3^^| ^ 1) (Ci+Ki-K.-l)/2_ 

The integral of Vi over the Ui range is bounded up to constant by 

P2 = (1 + |ti|) (-3'*i+^+l)/2 (1 + |t2|) ('R+2Ci+2ki+K2+2)/2 
x(l + |ti+t2|)(-=^'=^-3'^^+^+^)/l 

Integrating P2 over |tj| < T^"*"^ gives the claim. D 

Finally, we give a 

Proof of (4.8). 

As before, we have 

|x|JJj(ki,K2; 2/1, 2/2)1 <«;i,«2,iJ,£ 2/i~'''2/2^''' // V dtidt2, 

\tl\,\t2\<T^+^ 

where V = 'Pki,k2,-r(^1)^2) is the polynomial factor 

p = (1 + |ti|) (-3«i+i?+l)/2 ^^ ^ i^^i^) (3«2+i?+l)/2 ^^ ^ 1^^ ^ ^^1^) (_3Ki-3K2+i?+l)/2_ 
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(Note that the exponential terms exactly cancel.) Integrating V gives the claim. D 

4.2. Estimating Py^. 

We use the bounds of the previous section to give an estimate for Pj,^. Among other 
things, we must verify that the inverse Lebedev-Whittaker transform pt,r satisfies (3.2). 
This will follow from the bound (4.19). 

By Lebedev inversion (2.8), we define 

^-.«(^) '-=7:^2 pW^^WM- — -^^- — r- (4-10) 

(0) (0) 11 r [-fj r [-^ 

Recall the double inverse Mellin transform formula for the Whittaker function (2.5), and 
that Wy{y) = W^^{y) for u tempered. 

Then putting (2.5) into (4.10) and comparing with (4.3) gives 



vGtt 



3 



r{l) 



Prjy) = j-^-^:R{Ci,C2;yi,y2), (4.11) 

and the equality holds for any Ci,C2 > 0. An immediate application of (4.6) proves that 
for any yi,y2,Ci,C2 > 0, 

P^yuy^) «c,,c.,n,s vmT'''^''''' {^Y"' (l)^''^^^- ^^.12) 

4.2.1. Pull past one set of poles. 

The above bound is insufficient for our purposes, so we return to the definition of X^jj, and 
pull the S2 integral from the vertical line (C2) with C2 > to the vertical line (— C2), with 
C2 = —C2, < C2 < 2. In so doing, we pass through simple poles at S2 = — cti, —02, — «3 
(generically the Oj are distinct). Then we can write 

p^^ = aJt + 7^l + 7^2 + 7^3, (4.13) 

where OJtis the remaining 4- dimensional integral (that is, a constant times X^}j(Ci, —C2; 1/1,1/2)), 
and the TZj are the 3-dimensional contributions from the residues at S2 = —o.j. Note that TZi 
is exactly equal to a constant times Xj,)j(0, 0, Ci; yi, 2/2)- In this integral, we pull the h'i^h'2 
integrals from the vertical lines with ^zvj = to the lines (^i), (^2) respectively, so that it 
becomes Xj.'^{ki, K2, Ci; yi, y2)- To ensure that we haven't passed any new poles, we require 
that the k's satisfy: 

|K,| < ^ (4.14) 

Recall that here, as always, the n' are related to n by (4.4). 
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The estimate (4.7) bounds 7?.i by 
whereas the term 971 is dominated using (4.6) by 

!/■!/. r»/»»/'^ (I)-"' (l)'V'. 

To make these the same in 1/2, we would hke to take n'^ = 2fi:i + k,2 as large as C2, subject to 
(4.14), which requires —ki + K2 < Ci and —ni — 2^2 < C2. This is easily achieved by, say, 
setting K2 = 0, Ki > 0; then we can take ni as large as 1, so that n'l can be as large as 2. 
We can take ni as large as 1, as needed, since R = 10. So under these conditions, we have 
dominated T^i by the bound we already have on Wl. The same can be done with 7^2 and 
7^3, by pulling k's to different ranges. 

We have thus given our second intermediate bound: for any yi,y2,Ci > 0, and any 
< C2 < 2, we have 

PrAyuy2) «C.,C.,R,s yW^r"'^'''^ i^^Y''' (f )'^T^ (4.15) 

By symmetry, we have the same result with the subscripts "1" and "2" reversed. 

4.2.2. Pull past two sets of poles. 

The above is still insufficient, so we return to (4.13). In the 9Jt integral, we now also pull 
the si integral from (Ci) with Ci > to (— Ci), where Ci = — Ci, with < Ci < 2, passing 
through poles at si = aj, giving 

Dn = m + ni + TZ2 + n3- 

The integral dJl is exactly equal to an absolute constant times Ij. j^{—Ci, —C2;yi,y2), and 
hence we apply (4.6), giving 

The integrals TZj are of the same form as Ij. j^{0, 0, C2; 2/1, 1/2), and after pulling to appropriate 
k's, we can dominate the TZj integrals by DJl, exactly as before. 

In the integral T^i, which is exactly equal to a constant times Xj. j^{0, 0, Ci, yi, 7/2)5 we can 
pull the Si integral from (Ci) to (— Ci), passing through poles at Si = 02,03. Hence we can 
write correspondingly 

Here T^.'^ is a triple integral, exactly equal to a constant times Xp^(0, 0, — Ci; yi,y2), and "Pi, 2 

is a double integral, exactly equal to a constant times Xrp'j^{0,0;yi,y2). The term ^1^3 is 
similar to Pi 2- 



SYMMETRY TYPES OF GL(3) FAMILIES 21 

In the double integral Pi 2, we can pull contours in Uj to any (kj) with 

IkjI < (2 + i?)/3, (4.16) 

without passing new poles. Apply the estimate (4.8) to bound Pi 2 by 



2/12/2 



r"--'^(|)""'(|)"''r-. 



|k,| < -^^ (4.17) 



Elementary linear algebra shows from (4.4) that if we choose 

then — k'^ = Ci and K'2 = €2- Since < Ci,C2 < 2, the condition i? > 10 is more than 
sufficient to ensure that (4.16) is satisfied. So the ^1^2 contribution is dominated by that 
from VJl. The same (by a different pull in k's) holds for Vi^^. 

Lastly, consider the triple integral 7l[ = Irpj^{0,0,—Ci;y 1,1/2). Since — Ci = Ci < 0, the 
next poles in u arise when ^2, ^3 = 2 + Ci. Hence we can pull the z/ variables to any ki, K2, 
satisfying 

R + 2 

y 

^2' '^s ^ 2 — Ci, 
without passing more poles. The estimate (4.7) bounds 7l[ by 

.■..r««/^(|)"(f)"'r-. 

Again, taking ^2 = and ki > 0, the second inequahties in (4.17) are satisfied, and we can 
take Ki = C2/2 < 1, so that k[ = €2- There are no new constraints on R. 

Hence we see that the contribution by T^i is dominated by that of 9Jt. The same holds for 
7^2 and TZ^ by symmetry, and we have established the following crucial bound. 

Proposition 4.18. Fix R> 10 and e > 0. For any yi, ^2 > 0, T ^ 1, and any < Ci, C2 < 

2, we have 

PrAyuy2) «C.^C.,R,e ymT'"^''"' C^f (D'^T^ (4.19) 

In particular, Pt,r satisfies (3.2), as needed in the Kuznetsov formula. 

Remark 4.20. One can pull further and analyze contributions from higher poles. In so doing, 
the power in T increases, so that the residual contributions TZ dominate the contribution 
from 9Jt. It may still be possible to get further improvements from such an analysis, but the 
above is sufficient for our purposes, so we stop here. 
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5. Bounds for the Kloosterman and Eisenstein Contributions 

Since we showed in the previous section that our choice pt,r of test function satisfies the 
requisite bound (3.2), we now invoke Kuznetsov's formula with this choice, and estimate the 
resulting components. 

5.1. Bounds for the Kloosterman integrals J' and J'. 

We shall apply the estimates obtained in the previous section to bound the Kloosterman 
integrals JT" and J' defined in §3. We begin with an analysis of the more difficult case of JT". 
For ei, €2 G ±1, recall Jlei,e2 is given by 



{A1A2) // /// e{-eiAiXiyi - e2A2X2y2) Pt ni^iVi^ ^2y2) 

J jRl J J Jr3 

A2 X1X3 + X2 Ax a;2(a;iX2 - 0:3) + x\ 



xe 



y2x\^ x\^\ yi (xia:2 - Xs)^ + x\ + l 



xp 



A2 \/{xxX2 - X3)2 + x^ + 1 Ax \Jx\ + a;| + 1 

2/2 Xl + X^ + 1 



y\ (X1X2 - X3)2 + X? + 1 



dx 



dy_ 
y ' 

(5.1) 

Here dx = dxidx2dx^. 

We put absolute values inside the integral and note that the resulting bounds are then 
independent of ei, e2, so it is convenient to drop ei, €2 from the notation. Since p = Pt,r, it is 
convenient to recall the dependence of JT on T and R, so we write henceforth Jt,r{Ai, A2), 
etc. We have 



UtA^I^ ^2)1 < 



{A,A2 




bT,fl(^l2/l'^22/2) 



Pt,r I ^ ' ^ 

\ 2/2 6 yi 6 



dx 



dyidm 
yiy2 



^2 = 1 + X2 + X3. 



where 

^1 = 1 + Xi + (xiX2 - X3)^, 

For i = 1,2, break the y^ integrals according to y^ > 1 or ?/j < 1; this gives 

Jt,r <Jo + Ji + J[ + ^2, 
where the y integral in Jq has 2/1,2/2 < 1, the J7i, J[ integrals have one bigger and one smaller, 
and J2 has 2/1,2/2 > 1- 

We first estimate Jq. Apply the bound in (4.19) to the second appearance of p^,^, choosing 
the largest possible values Ci = C2 = 2 — e: 



Jo <£ 



2^9/2+3iJ/2+£ ( AxA 



(A1A2) 



2^2 



2-£ ^1 



bT,fl(^iyi'^22/2)|- 



v4iv42 



io io 



JO (yi2/2) 

1 '•^ lp^_^(Ai2/i,A22/2)U2/irf2/2 



3-e 




m-i)"^ 



dx 



dyidy2 
yi2/2 



(yi2/2; 



3-e 



2/1^2 
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since the x integral converges absolutely. Now estimate the first p^ ^ again using (4.19), with 
Ci = C2 = 2 - e/2: 



dyidy2 



{yiVif " yiy2 



Jo 



yiy2 



since the ^/-integral converges absolutely. 

To bound J\^J[^ and 1/2, we simply follow the same procedure as above with minimal 
changes to ensure convergence, as follows. For J72, in the second application of (4.19), we 
choose Ci = C2 = 2 — 26, say, so the final y-integral converges absolutely. Similarly, for J\ 
and J7i', we do the same as before, except in the second application of (4.19), we choose 
Ci = 2 — e/2, C2 = 2 — 25 (or vice versa), so that the y-integral converges absolutely. We 
have thus proved that 

jr,/?(Ai,A2) «ii,. ri+3^+^(AiA2)^-^ 



Next, we want a similar bound for Jt,r{_-^- Recall that we have 
JTfi[A) < A-' [[ [[\p^^{Ay,,y,)\ 



Ptm 2/2 



a/1 + xf + X[ 



A 



l + xf 



xf 



yiy2 1 + xf + x^ 



, , dyidy2 
axiax2 5—. 

ym 



Note that here the integral involves dy2/y2, whereas in J' the integral has dy2/y2- This will 
result in a weaker final bound. 

As before, for i = 1, 2, break the yi integrals according to yi > 1 or yi < 1; this gives 



Jt,b^ <Jo + Ji + J[ + J^. 



2, 



where the y integral in Jq has yi, 2/2 < 1, the J7i, J[ integrals have one bigger and one smaller, 
and J2 has 1/1,1/2 > 1- 



We first bound Jq. Set ^1 = 1 + x^ and ^2 = 1 + a;i + x^- Replace the second p^^ by its 



bound in (4.19), with the choice C\ = e and C2 = 2 — e: 



A2 



^0 



2-e 



. «. i!^ r r / /i...(-.^.^.i-4f;(^)X^)^"^--1: 



dy2 



^ rpb/2+?,R/2+e ji^l-e I I 

Jo Jo 



2/1 V6 

—00 —00 

^ '"^ bT,H(^2/i'2/2)| dyidy2 



yivl 



(2/12/2) 



3-e 



2/12/2 
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since again the x integral converges absolutely. Here we used that for fixed Xi > 1 and 
Z < -1/2, 



[1 + xf + xiy dx2 <^x[ 

Now apply (4.19) to p^. ^{Ayi, y^) in the above integral with Ci = C2 = 2 — e/2. It follows 
that 



Jo « rV2+3H+.^i-= /" /" j^y_^y^j.iii+mi2+, (Mm2 



Since £ > 0, the above jz-integral converges, and we obtain the bound 

Jq < 2-3+3/?+e^4-e_ 

Then we bound Ji^ JT/, and J72, by the same procedure as above, with suitable modifica- 
tions, as before. We record the previous computations in the following. 

Proposition 5.2. Fix R > 10, and any small e > 0. 
For any Ai,A2>0 and T ^ 1,, we have 

JtA^i, A2) «ii,. Ti+3i?+.(AiA2)^-^ (5.3) 

For any A > and T ^ 1, we have 

JtA^) <^R,e T^+^R+^A'-\ (5.4) 

5.2. Bounds for the Kloosterman contributions. 

We shall now apply the bounds in Proposition 5.2 to estimate the Kloosterman contribu- 
tions /C,AC, and /C^ appearing in the geometric side of the GL(3) Kuznetsov formula (3.6). 
We need estimates for these contributions using our choice of test function (or its transform) 
given by (4.1). 

Let us begin by bounding the long element Kloosterman contribution /C. We only use the 
trivial bound for the Kloosterman sum: 

|^(mi,ni,m2,n2,Di,D2)| <. (^^1/^2)'+^ (5.5) 

It immediately follows from Proposition 5.2 and the trivial bound (5.5) that 



^ ^^ ^ ^ d;d2 ^^'« [ D2 ' D, 



Di = lD2=l 

00 00 

|£-2 



Di=lD2=l 

< |mim2nin2pTi+^^+^ (5.6) 
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Next, we obtain a similar proposition for the lower rank Kloosterman contributions /C and 
Ky . In this case, we have the trivial bound 

5(mi,ni,n2,/^i,/^2) <e {D^D^f^'. 
It immediately follows from Proposition 5.2 that 



««E E 



\S{mi,ni,n2,Di,D2) 



D1D2 

D2=l 01102 

'"2-Di="lD2 



J, 



T,R 



nin2mi\ 
D1D2 J 



00 

2e-2 



«K,. Immm^pT^+^^+^J^ 5^ \D,D2 



D2 = l Dj^\D2 

m2Df = niD2 

< |mlnlr^2|^T3+^■^+^ (5.7) 

The same argument applies to /C^. We have proved the following. 

Proposition 5.8. Fix R > 10, T ^ 1, and e > 0. Then the Kloosterman contributions JC, 
/C and Ky in (3.6) satisfy the hounds: 

IC <R,, |mlnln2|'T3+3^+^ 
/C^ «fl,e |m2n2ni|2r3+3«+^ 

5.3. Bounds on the Continuous Spectrum. 

Next we obtain bounds for the terms £min, £max coming from the continuous spectrum in 
the Kuznetsov formula (3.1). We begin with the term coming from the minimal parabolic 
Eisenstein series: 

joo ioo u 

1 f [ |p^^(z/i, 1^2)1 

£min = ,, ..^ I I A^{ni,n2)A^{mi,m2)— ■ diyidu2, (5.9) 

-ioo-ioo 11 |C(l + 3z/fc)i (^2 jl 

fc=i 

where the minimal Eisenstein coefficients satisfy (3.4). Inserting the choice of test function 
(4.1) into (5.9) and using the de la Vallee Poussin bound (Prime Number Theorem) 

IC(i + ^t)|» 



log(2+|t|)^ 
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we get from Stirling's formula (4.2) that 



^min **^ 



{Ani) 



mim2^in2 



n 



-p I 2+-R+3I/J \ -p ( 2+R-3iyj_ 



iT^+e _irp\-\ 



fc=l 



duidu2 



< 7-2+3R+e |,72^,7^2nin2|^. 



^, 



Next we consider the term coming from the maximal parabolic Eisenstein series: 

c ST^ I ni,,rAni,n2)ni,,rA'ri'i'i/rn2) Pt,r\^ 3"' ~ 

2Txi 



E 



Bu,rj {ni, n2)B^^rj {mi, 7712) 
L{l,Aduj)\L{l + 3u,Uj)\'^ 



(5.10) 



du. 



-p I l+'Su—irj \ -p I l+2irj \ -p ( l+3v+irj 



where c is an absolute constant, {uj} is a basis of Hecke-Maass forms for GL(2,Z) (each 
of eigenvalue 1/4 + r|), and the Fourier coefficients satisfy (3.5). Here we have the lower 
bounds 

L{l,Aduj) >e (1 + \rj\)-', L{l + 3u,Uj) >e (1 + \u\ + \rj\)~'. 
These lower bounds follow from [HL94, HR95, JS77] and [GLS04]. Combining the above 
lower bounds with Stirling's formula (4.2), it follows that 



iTl+e 



^ J L(l,Adu^ 



-p f 2+R+3u-irj 



,«Ti+=_.yi+, 



)|L(l + 3z/,n,)P 



r 



2+R+2ir. 



dv 



ij,l+e 



■C |mim2nin2|^/^"^^ y^ / (l 



|3z/-irj|)^'*- (1 + |rj|)^'^ |rfz/| 



r,«Ti+-_.^l+, 



< |mim2nin2|^/2+" 2-3+3i?+e 



(5.11) 



using Weyl's Law for GL(2) and the Ramanujan conjectures at infinity (for GL(2)). In 
summary, we have proved the following. 

Proposition 5.12. Fix i? > 10 and e > 0. For any T ^ 1, we have 
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6. Proof of Theorem 1.3 
6.1. The Main Term. 

We begin by computing an asymptotic formula for the main term Ai in the Kuznetsov 
formula (3.1), (3.6). It follows from (2.10) and Stirling's asymptotic formula (4.2) that the 
inner product in the main term (3.6) becomes 

joo joo 
{Pt.r^ Pt.r) = J—T^ / / bT.if(^l>'^2)P .2\^ ( ^u \ ^^^^^ ^^-^^ 

[txiY J J rr r(^)r(^ 

-loo-joo l<j<-i \ ^ V 



~c' 



T-r -p f 2+R+Zvj \ -p ( 2+R-iVj 
l<i<3 ^ ^ 



2 



n r(|^)rte 

1<7<3 V / V 



-dvidv2 



for some constants c, c' > 0. This gives the T^+^^ main term as claimed in (1.4). 
6.2. Completion of the proof of Theorem 1.2. 

It follows from the Kuznetsov formula (3.1), together with the choice of test function 
P = Pt R (with p*j^ given by (4.1)), that 

r \^ A t \'n shrfiii'i ,1^2 ) 

C = 2_^Aj{mi,m2)Aj{ni,n2) -^ 

j ^ 

= M+}C + }C + }C'^- Smin - Smax, (6.2) 

with M = lrni=™a {Pt,r, Pt,r)- 

Then Theorem 1.3 is an immediate consequence of the estimates 

\Pt,r^ Pt,r/ ~ '^-' 
\Smin\, \£ma.\ <e \m^ni2n^n2\^'^^' ^Z+ZR+e 

|/C| + 1^1 + l^^l <ij,, |mim2nin2pT3+3«+" 
given in (6.1), and Propositions 5.12, 5.8, respectively. 
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7. The Explicit Formula 
For a Hecke-Maass form on GL(3), let p(0) be one of 



P(0) = { sym^ 
Ad0, 

and let L{s,p{(f))) be the corresponding L-function. Note that the Maass form dual to </> is 
just the complex conjugate (p. In view of the identity 



L(s,Ad( 



L{s, (f) X 



it is easier to work with the Rankin- Selberg convolution of and instead of the adjoint 
L-function. 
Define 






^nr(^)^'nnfi 



fc=i 



p k=\ 



ak{p) 



A{s, p(0)) := <^ 



_5s 
IT 2 



n r(^±^)-^n n u 



n-'^fl n r (^±^^ 



-1 



3 3 



)~ nn n 1 



"j(p)"fc(p) 



"i{p)afe(p) 



if p(0) = 0, 



if p(0) = sym2 0, 



if p(0) = 0X0. 



j=i fc=i p j=i fc=i 

Then, in all the above cases, we have the functional equation 

A(s, p(0))=A(l-s,p'(0)), 

where tt is the contragredient representation of tt; its L-function has Dirichlet coefficients 
which are complex conjugates of the original. This follows from [GJ72, BG92, JPSS83], 
respectively. 

We shall use the functional equation for A(s, p(0)) to determine the so-called "explicit for- 
mula" relating zeros and poles of A(s, p(0)) with sums over prime power Fourier coefficients 
ofL(s,p(0)). 

Let G be any holomorphic function in the region —1 < 9^e(s) < 2 satisfying 

G(s) = G(l-s), \s^G{s)\'^l. 

Let Pi = ^ + i'ji {i = ±1,±2,...) run over the zeros of A(s,p(0)) with corresponding 
multiplicity. As we have assumed GRH, the ordinates form a real increasing sequence 



< 7-2 < 7-1 < < 7i < 72 < ■ ■ 
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A' A' — 

^(S,P(0))+^(S,P(0)) 



(7.1) 



By the functional equation and standard shifts of contours, together with the fact (first 
proved by [BG92]) that -j^ has at most simple poles at s = 0, 1, with residue 

(O ifp(0) = 0, 
rp(0) = \ if p(0) = sym^ and not self dual, 
[l ifp(0) = 0x0, 

we have 

2+ioo 

5^G(p)-r,(^)(G(0) + G(l)) = ^ I Gis) 

P 2-ioo 

p<oo 

For a finite prime p < oo, the function Hp(^^-){p) is defined by 



ds (7.2) 
(7.3) 



oo / 3 



- E ( E [o^dpY + a,{py) F (/)) logp, 
e=i \k=i ^ ^ J 



if p(0) = 0, 



iip{<i»[v) =\~^\ S («j(p)^«fc(p)^ + «i(p)^afc(p)M i^ (/)) logp, if p(0) = sym^ 



tX) I ?> 3 



-2E EE«.(pf«fc(p)Mi^(/)iogp, 

Here -F(|/) is the inverse Mellin transform of G(s), 

1 



if p(0) = 0X0. 



i^(2/) = ^ / G{s)y-'ds. 



(1/2) 



For p = oo, we have that ifp(<^)(oo) equals 



3 2 



+ioo 



'^^ 1 5~*°o 



-3F(l)log7r+^E / f (^) + f (^) G(^)^^, 



if p(0) = 0, 



+JOO 



5F(l)log7r+^ E / f (^±¥^) + f (^^T^) G(3)rf3, ifp(0) = sym 



l<j<fc<3 i_,^ 



3 3 i+ioo 

-9F(l)logvr + ^EE / f (^±T^) G(.) d.. 



if p(0) = 0X0. 



j=\ k=\ 1 

2 
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Fix an even test function ip of Schwartz class whose Fourier transform has compact support, 
and apply the above formulae making the choice 



G{s) = i^[{s- 1/2) 



logCV 

27rz 

where we recall (see (1.12)) that Cp is the weighted average value of the conductor of 
Then 



y/y log Cp \hgCp 



It follows that 



Hp(ct>){oo) 



3-lp{0)logTT 

log Cp 



5V>(0)log7r 
log Co 



3 oo / r' f nix I i+2"fc \ , r' f ttl 



+ E / 

fc=l— oo 



r yiogCp^ 4 y^ryiogCp^ 4 

21ogCp 



ip{x) dx, p{(j)) = (f), 



+ 2iogCp E J \ r 



l<i<fc<3 -oo 



l+2(Qj+afc) 



log Cp 



r I log Cp 



+^ I t£^ + i^%±^ ) ) ^{x) dx, p(0) = sym2 0, 



9»/)(0)log7r 

log Cp log 



3 3 oo 



■1 = 1 k=l—oo 



T' I nix I l+2(aj-afc) 



log Cp 



Recall that 



D(p(</.); V') 



log a 



5^ ^(^^ 2vr 

A(f+i7»,pW)=0 



V'(a:) (ia;, p(0) = cf) x (j). 



(7.4) 



Consequently, (7.2) becomes 
B 



■sp \p -r / ^ logp \ logp v^ "fc(p) +Qfc(p)'^ 
pW Z^ Z^ ^ llogCp / logCp ^ 

p<oo i>l \ ^ f / '^ f 



3 

E 
fc=i 



;372 ) 



P(0) = 0, 



D(p(0); tA) = <^ 



bk^) - e e ^(gf ) i^ E "'(^)^"-(^);+r-^^^^"-^^^ p(0) = sym2 0, 

p<oo £>1 V * ''/ « ^ l<j<fc<3 



5pw-2E E ^ ( f^) iS^ E E ^^^^^'^^^^ 



p<oo e>i 



log Cp / log Cp 



j=l k=l 



e/2 ! 



P(0) = </> X (/). 

(7.5) 
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with 



and where 



Bp(^) = 2rp(^) V 



logCp\ , A 



V(0) 



Alii J log Cp 



(7.6) 



A 



pW 



3 oo 



-3V^(0)log7r + iE / 



fc=l— oo 



r \ log Cp 



+ J 



r' / Trix I l-2a 



!Co 



+ 



?/^(x) dx, p{(f)) = (p, 



-5^(0)log7r + | E / T 



l<i<fc<3 -oo 



r' / TTix I l+2(aj+afc) 



log Cp 



;C„ 



4 / 



+ ^fT:^ + ^^%^^'))V^(x)rfx, p(0) = sym2 0, 



3 3 oo 



-9^(0)logvr+EE / T 



V I TT'ix I l+2(aj-Qfc) 



j=l fc=l— oo 

Now, for *He(a) = 0, we have 

oo 



log Cp 



ip{x) dx, p((f>) = 0X0. 



r Vb^ + a + ^ J V^(x) rfx = 7^(0) - (^a + - j + O (^(1 a + 1/4| log Cp) 

— oo 

If we combine this with the fact that ^ ("^ "'" i) ~ ^oga + 0{1) for |a| > j, it follows that 



A 



^(O)loj 



|Qfcl 
2 



7r-3. n 

l<fe<3 



c?(i), 



p(0) = (p, 



( 



pW = S 



^(O)loj 



\ 



vr 



Qj+afcl 



n 

i<i<fc<3 



+ 0{1), p(0) = sym^ 



( \ 



V'(0)log 



3 3 



"i-"fcl 



^-'- nn 
i=i fc=i 



+ 0(1), p(0) = 0X0. 
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Thus, in all cases, we have 

^,(^) = ^(0)logc,(^) + O(l), (7.7) 

where the analytic conductor Cp(^) is given by (1.10). 

We combine the above formula for ^p(<^) with (7.5) and (7.6). The contribution to (7.5) 
from £ > 3 is negligible (using the Ramanujan bound |aj(p)| < 1), so we have 



D{p{<j>y, ^l^) = m - S;(^) - Sj(^) + 2r,(^) ^ 



log Cp{<p) 

Airi 



O 



log CpW J ' 



where S^/ ,n is the contribution from i = 1,2, namely 



(7. 



2- ^ \logCpJ logCp 
P<CXD ^ ■' k = l 



logp ^ logp y^ «fc(p) +afc(p) 



;tj2 , 



^pW - ^ 



E ^ 



P_\ logp 



E 



"j (p)Qfc (P) + "j (p)Qfc (P) 

„l/2 • 



logCpy logCp 



Q o 

9 Y^ „/, / logp A logp V^ V^ Qj(p)Qfc(p) 



p<oo 



log Cp / log Cp 



if p(0) = 0, 



if p(0) = sym^ 0, (7,9) 



if p(0) = 0x0, 



i=i fc=i 



and 



^2 



V Ji^ f 2 logp A logp v^ «fc(p)^+afc(p)^ 
Z^ '/^ I logCp^ logCp Z^ D • 



p<oo 



if p(0) = 0, 



fc=l 



S-1... = <: - 2. ^ i gg) l^ ^ ^E ^„ ^..(.)-^.(p)-^^..(p)-^.(p)- ^ ifp(0) = sym^0, (7.10) 



E ^ 

p<oo 



l<i<fc<3 



-2 E^ 

p<oo 



3 3 

EE^ 
i=i fc=i 



2 logp '\ logp V^ V^ aj(p)2Q:s,(p)2 
log Cpi log Cp 



if p(0) = 0X0. 
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8. Local Analysis 
Recall the Fourier expansion of </>: 

7GC/2(z)\sL2(z)fci>ifc27^o ^' ^' yy \j ^ ■' I 

where is normalized so that A^{\^ 1) = 1. Then the Fourier coefficients A^{k\^ k^) satisfy 
the Hecke relations 

A,{n^)A,{k,M) = E ^^(t^'T^)' (8-2) 



dQdid2 
dll*:i,d2|fe2 



as well as the conjugation relation 



A{kuk2) = A{k2,ki). 
Furthermore, the L-function associated to is given by 

" A^{l,n) 



p 
with local factor 



n" 

n=l 









It follows that 

A4l,p^)= Yl «1(P)"«2(P)^'«3(P)'". 

u+v+w=m 

In particular, 

A<^(l,p) = ai{p) + a2(p) + a3(p), 
and 

A^{l,p'^) = ai{pf + a2{pf + a3(p)^ + ai(p)a2(p) + a2(p)«3(p) + ai(p)«3(p) 
= ai{pf + a2(p)^ + a3(p)^ + A^{l,p). 

Next, we analyze the local contribution at a prime p occurring in the sum S)^/ ,^ given in (7.9). 
There are 3 cases to consider. In the second and third case we apply the Hecke relations 
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(8.2) to remove the product of two Fourier coefficients: 



^ak{p) = A^{l,p), 



k=l 



^ aj{p)akip) = A^{l,py - ai{p)a2ip) - ai{p)as{p) - a2{p)a^{p) 
= A^il^pf - A4i;p) = A^{l,pf - A^ip, 1) 



A^{l,p' 



3 3 



^^aj(p)afc(p) = A^{l,p)A^{p,l) = 1 1 + A^{p,^. 
j=i k=i 



Putting the above identities into (7.9) yields 









p<oo 



E ^ 



p<oo 



p \ logp A^(l,p^) + A^(p^l) 



log Cp/ log Cp 



,1/2 



-2 V ^ f i2g£_^ logp 1+A^{p,p) 
^ ^ \\ogCpJ log Cp pi/2 ' 



p<oo 



if p(0) = sym^ 



if p((/.) = 4>x ^, 



1.3) 



Next, we do the same for S^/ ,n given in (7.10). As before, there are three cases to consider. 
We require the following Hecke relations obtained from (8.2): 



A^{p, If 

A^{i,p'r 

A^{p,l)A^il,p^) 

/l<^(p^l)A^(l,/) 

-A^ip^l)A^ip,l) 

-A^{l,p'')A^il,p) 

A^{p,l)A^{l,p) 



A^{p^,l) + A^{l,p), 

A^{p^l) + A^{p,p^) + A^{l,p''), 

A^{p,p^) + A^{l,p^), 

A^{1,1) + A^{p,p) + A^{p^,p^), 

-A^ip,p)-A^ip\l), 

-A^ip,p)-A^{l,p^), 

1 + A^{p,p). 
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It follows from the above Hecke relations that: 

3 



Y^MpY = A^{l,p^)-A^{p,l), 



fc=i 



J2 aj{pfak{pf = {A^{l,P^) - A^ip, 1))' - A^ip"", 1) - A^{p, 1) 

= A^l.p^ - 2A^{p, l)A^{l,p') + A^{p, 1)2 - A^{p^ 1) + A^{p, 1] 



A^{p\ 1) + A^il.p") - A^ip.p") - 2A^{l,p^) + A^{l,p) + A^{p, 1), 



3 3 



3=1 k=l 

= A^{p\ l)A^{l,p') - A^{p, l)A^{p\ 1) 

- A^{l,p')A^{l,p) + A^ij), l)A^{l,p) 



2 + A^{p\p') - A^(l,p^) - A^{p\ 1). 



Putting the above identities into (7.10) yields 



^ ^ \ logCpJ logCp 



p<oo 



p{4>) = <P, 



^ ^ yiogCpjiogCp 

p<oo 



2\ogp\ logp I A^{p'^,l)+A^il,p*)-A4,ip,p^)-2A4,il,p^)+A^il,p)-A^ip,l) 



^ A^il,p^)+A^ip\l)-A^(p',p)-2A^(p-\l)+A^ip,l)-A^il,p) \ ,,. ^ 2 j, 



y2 



(8.4) 

Remark 8.5. It is much simpler to apply the Kuznetsov formula to the sums S^.,-,, SV^ 
given in (8.3) and (8.4) since the product of Fourier coefficients has been removed by the 
use of the Hecke relations. 



-2 E^ 

V. p<oo 
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By Parseval's Theorem, 



9. Proof of Theorem 1.13 



^p{x)Wp^^){x)dx = / %l){y)Wp(^){y)dy 



where ip is the Fourier transform of ip, and Wpi^^) can be exphcitly computed from (1.15) as 
a distribution: 

'5o(y), if p(0) = or sym^ 0, 

1/2, \y\<l, 

^o(y) -<i 1/4, |2;|=±1, if p(0) = Ad0. 

|2/I>1,- 



W^pW(z/)= < 



As the support of ijj will be restricted well inside (—1,1), it follows that 

_ r^(o), ifp(0) = </', 

^(l/)W^pW(y)t^y = S V^(0), if p(0) = sym2 ^ 

[V^(0) - i7A(0), ifp(0)=Ad0. 



(9.1^ 



Recall the asymptotic formula (7.8) for the low-lying zeros sum: 



D{p{<P)- ^) = ^(0) - Sj(^) - Sj(,) + 2r,(^)^ f i^l^^ 



Ani 



+ 



log Cp(0) 



where by (7.1), we have 



(9.2) 



ifp(0) = 0, 
Tpi^^-) = < if p(</)) = sym^ and not self dual, 

1 ifp(0) = 0x0, 

and S^ ,w S^. ,N are given by (8.3), and (8.4), respectively. 

We will prove Theorem 1.13 where the limiting density j^ip{y)W {y)dy is in the form (9.1). 
Note that ip{Q) already appears in (9.2). 

The case when p(0) = cj) x cj): 

The main contribution to Si. i + T?, i comes from 






logp \ logp 



logCpJ log Cp 



*^IK 



21ogp\ logp 



log Cp J log Cp' 



(9.3) 



This may be computed in two steps. 
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Step 1: We apply the explicit formula (7.2) to the function 

A(.) = vr-f r (I) C(s). 
Note that for log Cp sufficiently large, we have 

Mp)=0 A(i+i7)=0 

since there are no low-lying (with I7I <^ 1/ log Cp) non-trivial zeros of the Riemann zeta 
function. Further G(0) + G'(l) = 2^ f °|^/ j. It follows that the explicit formula for the 
Riemann zeta function takes the form: 



2^ 



/ logCp \ _gv^ 1 7/ ^logp \ logP V^(0) logTT 
V im J 2^^pm-^{\ogcJ logCp + logCp 2 



+ - ip[x) ax 



hgCj.^T VlogCp 4 



^ p^/2 "^ ^ log Cp ; log Cp V log Cp 



Note that the above accounts for a large part of (9.3) and almost cancels the term 2?/; 
in (9.2). In fact, we get 

D(0 X 0; V^) = ^(0) - Sj,^ + 2^ (!^^) _ 2^ ^l°g^^ 









A-ni I \ Aixi 



-2E-*(f^)T^ + o(r^)- P*) 

What is really happening here is that every Rankin-Selberg L-function L(s, x 0) in the 
family is divisible by the Riemann zeta function which has a pole at s = 1. But the Riemann 
zeta function does not contribute low lying zeros, so the contribution from the pole at s = 1 
is cancelled. 
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Step 2: In the second step we make use of the classical Riemann hypothesis which implies 
that ^(x) = X + O [xs+^y It follows that 



2Er* 



1 ^ f2\ogp\ logp 



P \ log Cp J log Cp log Cp J I V log C. 



^(^}2l^]a:-'d^{x) 



^ I ^^ 1 X- 



log Cp J I V log Cp 
2 f°° 2 



x + O (x 



dx 



^' 



-, ( 2 log X 



^og Cp J I X log Cp \\ogC, 



X 



'^ (x + o(x^+')) dx 



°° - /I 



If we now combine the computation in Step 2 with equation (9.4), it follows that 



D{<f> X 0; ^) = m - 1^(0) + 2^ fl^SC^^ _ 2^ (^-^^ ] + S^ 



V Ani 



Ani 



4>x4> 







logc, 



4>x(i> 



where 



(9.5) 



y3 _ 



2E 



A4,{p,p) 7/ logp 



V 



+ 



Vp " ViogCp, 
^<^(p',p')-^^(i,p') 



p 



A^{p\l)\ 7/ 2 logp 
"^VlogC, 



logp 

i^gc^' 



(9.6) 



To prove Theorem 1.13 for the family of Rankin Selberg L-functions, we make use of (9.5) 
and the decay properties of hx^R to obtain 



2^ D{<Pj X 0,-; V^) ^ = 2^ 



1 

2'' 



^(o)-^^(o) + 4^.,^^. 



o 



log log T 
logT 






as we average over Maass forms 0j (j = 1,2,...). The term log log T/ log T arises after 
breaking the sum into two pieces corresponding to 0j with conductor c^. <^ T^/logT and 
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p. ^ T^/logT. To evaluate the above sum, it remains to estimate 



E^ 



ct>jXct>j £ 



Say the support of tp is in {—S, 6). It immediately follows from (9.6) that 



E^ 



hrA'^^'^) 



4'jy-<Pj £. 



2E 






^cfiP^P)? [ logP A logP 



^/P 



-i' 



logCpJ log Cp 






^<^(p ,p )-^0(i,r)-^0(p , l)^ ^/^2iogp^^ log J) 



p 



logCp/ logCp 






To finish the estimation, we apply Theorem 1.3 to obtain 

/lT,«(!'"') 



E^ 



(/)jX(/>j £. 



<T 



S+Sii+e 






p^TS* 



logC'p ^^3. log^P 



^ Tn3+3iJ+27(5+£ 



So we need 



<5< 



With this choice of 8 we obtain 



^B{<^j X0J-; V^) 



A 



27' 



,^(o,-l„o,.o(!2l||I 



■E 






as claimed. 



Remark 9.7. The family of Rankin-Selberg L- functions for GL{3) has the same symmetry 
type as the family of adjoint L-functions in view of the identity L(s, Ad0) = /(t) ? ^-^d 
the fact that ({s) has no low- lying zeros. 

The case when p(0) = sym^ (j) : 



It immediately follows from (9.2), (8.3), and (8.4), that 
D{sYm'^(j);'^)=i^{0)-j:l ^^ 

where 



sym- 



^sym 



'^' sym^ ( 



^ I lOgCsy^^ I ^ ^ 



^sym^0--E^(l^j 



47r2 / VlogCsym2 

logp \ logp A^{l,p'^) + A^{p'^,l) 



log a 



p 



1/2 
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and 



S2 



sym- 



->> 



2\ogp\ logp 



^JVogCjlogC, 



X 



A^{p\ 1) + A^(l,/) - A^{p,p^) - 2A^(l,p3) + A^{l,p) - A^{p, 1) 



P 



+ 



A^iiy) + A^{p\ 1) - A^{p\p) - 2A^{p\ 1) + A^{p, 1) - A^{l,p) 



P 



In this case rgyma ^ = unless is self dual, which happens only if is a symmetric square 
lift from GL(2). This occurs for x T^ cases out of x T^, and hence contributes a negligible 
error term to the low-lying zeros sum. The method to estimate S^ 2^^ and S^ a^, using 
Theorem 1.3, is exactly the same as the method used above for the case of p(0) = 0x0. 
Because of the presence of A^{l,p'^),A^{p'^, 1) in ^.l^^^^ and A^{l,p'^),A^{p'^, 1) in S^^^^a^, 
we obtain the same value (^ < ^ (which we obtained for the case p(0) =0x0) for the 
support of V'. It follows that 






c, 



^(0) + O 



log log T 
loffT 



E 






The case when p(0) = : 

In this case, we have that the residue r<^ = 0. It then follows from (9.2), (8.3), and (8.4), 
that 

^1 _ _^ \r^ 7 Ajogi^X logp A^(l,p) + A^(p, 1) 



where 



and 



p<oo 



p<oo 



21ogp\ 21ogp A^(l,p2)-A^(p,i) + A^(p2,l)-A^(l,p) 



logC^y logC,, 



p 



Assume the support of ip is in (—5, 5). As before, we have 



Ez)(fe*.)^^ = E 



m - Y.\ - EL + O 



log log T 
loffT 
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and invoking Theorem 1.3 again gives 



E i'^k 



p^T^s/ 



Ct 



E 



^ii 



p^T^s 



logp \ logp A^{l,p) + A^{p, 1) 



logC y logCp 



V 



1/2 



Y^ -/21ogp\ 



2\ogp\ \ogp A^{l,p^) - A^{p,l) + A^{p\l) - A^{l,p) 



logC, 



p 






^ 2^3+3iJ+£ 






E p' 



^^ T-i3+3-R+e 

So in this case, we need 5 < ^. 



T- 



This completes the proof of Theorem 1.13. 
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